Qualitative Evaluation Criteria for Parallel Programming Models

Programming parallel machines has always been difficult, but the new multicore reality has triggered significant research projects such as those part of the DARPA HPCS challenge that put forth programmer productivity as on par with performance. Since programming models are crucial in supporting the programming of parallel machines, this raises the question of how the PLDI community should evaluate programming models along the productivity and performance dimensions. Meaningful quantitative metrics are the ideal solution. Fortunately, performance metrics are abundant and relatively understood. More problematic is how to evaluate the effect of a programming model on programmer productivity. Existing programmer productivity metrics include things such as SLOC or development time. Other factors have been determined important but are difficult to quantify: readability, maintainability, portability, correspondence with serial code, adaptability of algorithm, etc. These programmer productivity metrics all contribute to the big picture, but have limited meaning and do not necessarily provide a clear direction for future programming model development. Due to the fact that parallel programming implementation details often obfuscate the original algorithm and make later algorithm modifications and maintenance difficult, we believe that parallel programming models should provide features that enable separation between algorithm and implementation detail specifications. A similar idea was suggested in the Parallel View from Berkeley paper [1], where they suggest programming models provide a separate interface for application programmers and parallel implementors. To encourage the separation of algorithm and implementation details, we propose programmer control and tangling as two qualitative measures for evaluating programming model constructs. We show example categories within the tradeoff space of programmer control and tangling, and we suggest a methodology for using these qualitative measures for comparing programming models. We argue that there is a rough correlation between tangling and reduced programmer productivity and between programmer control and performance. We use the term tangling to indicate that the implementation details are exposed in the algorithm code in an intermingled manner. In general, it is important that tangling be kept to a minimum while still providing the programmer control over the parallelization and optimization details for performance tuning purposes (i.e., programmer control). Interspersed